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DETAILED ACTION 

1. This Office action for US Patent Application 10/729,834 is responsive to the 
Request for Continued Examination filed 18 October 2008, in reply to the Advisory 
Action of 22 September 2008 and the Final Rejection of 18 July 2008. Currently, claims 
2-6 and 8-12 are pending. 

2. In the Final Rejection of 18 July 2008, claims 2-4, 6, and 8-11 were rejected 
under 35 U.S.C. 103(a) as obvious over "Motion Compensated Enhancement of Noisy 
Image Sequences" (Kalivas et al.) in view of International Publication 00/64167 A1 
(Prakash et al.). Claims 5 and 12 were rejected under 35 U.S.C. 103(a) as obvious 
over Kalivas et al. in view of Prakash et al. and in view of US 5,544,239 A (Golin et al.). 

Continued Examination Under 37 CFR 1.114 

3. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 18 
October 2008 has been entered. 
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Response to Arguments 

4. After reviewing the arguments presented in the Request for Continued 
Examination, it is believed that the Advisory Action related from a misunderstanding or 
misinterpretation of the arguments presented in the Request for Consideration filed 04 
September 2008. Previously, it was believed that Applicant was contending against the 
mapping of the limitation of "adjusting weights from color blur of said segments", the 
"said segments" being the "arbitrarily shaped segments" of the first element. Now it is 
understood that Applicant faults the alleged mapping of "color blur" of "segments that 
are no longer adjacent" (Arguments, 18 October 2008, pg. 4). For clarification, a 
complete mapping will be made of claim 2, as requested. 

A method for temporally filtering a video sequence, the method 
comprising: 

Kalivas et al., sections 3.1 and 4.1 , describe temporal filtering, 
using object motion estimation 

Kalivas et al., section 4, describes temporal filtering and spatiotemporal filtering 
with motion compensation. 

for arbitrarily shaped segments 

Kalivas et al. performs a segmentation between moving objects and a 
background (equation 3), and performs spatiotemporal filtering based on this 
segmentation (section 4). Additionally, Prakash et al., pg. 8, lines 27-34 states: "At step 
204, functional block 104 of encoder 100 segments i.e. obtains the structural 
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information from, the reconstructed image, the segments being used as the basis for 
prediction frames in terms of the kinetics (e.g. motion and/or residue data) of the 
segments.... Alternatively, step 203 is skipped and the encoder segments the original 
reference image frame from step 201" (emphasis added), 
to align corresponding pixels 

Kalivas et al., section 2.2, gives a motion model that "describes exactly the linear 
motion of the object pixels and gives a very good approximation in the case of the 
nonlinear motion". Pixels r at time k and r' at time k+1 are a pair of "corresponding 
pixels". 

between at least two frames; 

In Kalivas et al., the frames at time k and k+1 in a multi-frame image sequence 
(§5) are the "two frames". 

determining segments 

These are segments of an image according to the segmentation process in 
Prakash et al., first cited in the first element of the claim. 

that are no longer adjacent to a segment boundary 

In Prakash et al., a newly-exposed, or "previously hidden" region (pg. 9: lines 16- 
29) separates two previously adjacent regions. Prakash et al. describes processes for 
encoding data from previously-hidden regions as residue information, such as infilling. 
Interior portions of these previously-hidden regions are "no longer adjacent to a 
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segment boundary", or the new boundary between the previously-hidden region and 
another object. 

based on said object motion estimation; 

Coding of previously-hidden regions in Prakash et al. is made more accurate by 
using structural information of the segments such as boundary location (pg. 19: line 27- 
pg. 20: line 2) and the motion of the moving segments (column 9: line 30: column 10: 
line 8). 

reducing impact of color blur 

Kalivas et al. states that blurring of moving objects from temporal filtering and 
blurring of edges from spatial filtering "can be avoided or at least significantly reduced" 
in a spatiotemporal filter "if the motion of the moving objects and the positions of their 
boundaries are known" (section 4). 

from said segments that are no longer adjacent 

These are the previously-hidden regions in Prakash et al. in the second element. 
Remember that these regions are coded based on the motions and positions of 
boundaries of moving objects, as in Kalivas et al. 

by adjusting weights assigned to one or more frames 

Object indicator function A(i,j,k) of section 2.1 of Kalivas et al. is used in equation 
3 to segregate moving objects from a background region and equation 12 to ensure that 
only moving objects only undergo motion compensation, with A=1 for all pixels in moving 
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objects and A=0 for all pixels in the background. Since A is a function of time k, it is 
"assigned to one or more frames". 

for pixels that lie within a blur region near said segment boundary; 

Prakash et al. improves on Kalivas et al. by specially encoding previously-hidden 
regions and setting them to the background or motion as determined by a prediction 
(pg. 23: line 9-pg. 24: line 10). By properly determining whether a newly-exposed 
region is in the background or is in motion, the residue error near boundaries (column 
23: line 30) is reduced, since the pixels in these regions are now more accurately 
assigned values of A for the motion compensation spatiotemporal filtering of Kalivas et 
al. 

and computing a weighted average 

The spatiotemporal mean filter defined in Equation 16 of Kalivas et al., weighted 
via A(l,m,n), is a weighted average, 
of color values 

Conventionally, pixel values are considered to have luma (brightness) and 
chroma (color) values. 

of said corresponding pixels. 

The filters described in section 4 of Kalivas et al. account for motion 
compensation, and so use the "corresponding pixels". 
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In summary, the first element of the claim was based on Kalivas et al. The 
second element of the claim was based on Prakash et al. The third element of the 
claim was based on a combination of Kalivas et al. and Prakash et al. to improve 
efficiency and reduce error of the filtering system of Kalivas et al. by using the special 
coding techniques of Prakash et al. to determine if a previously-hidden pixel should be 
included in motion compensation when performing a spatiotemporal filter. The fourth 
element of the claim was based on Kalivas et al. 

5. Applicant's arguments filed with respect to claims 4 and 11 have been fully 
considered but they are not persuasive. Applicant states that in MPEG, a frame 
"referencing" a second frame is distinct from a frame determining motion information or 
performing motion estimation from a second frame. With all due respect, this statement 
is in error, since in MPEG video, reference frames are used for motion prediction. See 
for example US 5,650,823 (Ngai et al.) column 5: lines 2-6: "Thus one would require 
three motion estimations to find a closest match for a B picture: one motion estimation 
with each of the two reference pictures and one motion estimation with the interpolated 
reference"; US 5,619,268 (Kobayashi et al.) abstract: "A motion estimation method and 
a motion estimation apparatus for calculating a motion vector to estimate a current 
picture partially forming a video sequence on the basis of first or second reference 
pictures partially forming the video sequence"; or US 5,585,862 (Wuertele et al.) column 
1: lines 4-10: "The present invention relates to a motion estimation apparatus. ..for 
calculating a motion vector to estimate a current picture partially forming the moving 
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picture on the basis of a reference picture partially forming the moving picture". In an 
Open GOP, this motion estimation from a reference picture may be performed from a 
reference picture not in the present GOP, such as the first I frame of the subsequent 
GOP, as described in the Waggoner reference. 



Claim Rejections - 35 USC § 101 

6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

7. Claims 2-6 and 8-12 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. Supreme Court precedent 1 and 
recent Federal Circuit decisions 2 indicate that a statutory "process" under 35 U.S.C. 101 
must (1) be tied to another statutory category (such as a particular apparatus), or (2) 
transform underlying subject matter (such as an article or material) to a different state or 
thing. While the instant claims recite a series of steps or acts to be performed, the 
claims neither transform underlying subject matter nor positively tie to another statutory 
category that accomplishes the claimed method steps, and therefore do not qualify as a 
statutory process. In the present invention, it is nowhere stated in the claims what 
apparatus performs the claimed method steps. 



1 Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalk v. Benson, 
409 U.S. 63, 70 (1972); Cochrane v. Deener, 94 US 780, 787-88 (1876). 

2 In reBilski, 88 USPQ2d 1385 (Fed. Cir. 2008). 
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Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 2-4, 6, and 8-11 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Motion Compensated Enhancement of Noisy Image Sequences" 
(Kalivas et al.) in view of International Publication 00/64167 A1 (Prakash et al.), 
admitted as prior art in page 7: line 34-page 8: line 12 of the specification of the present 
invention, and cited in the Information Disclosure Statement of 18 June 2004. 

Kalivas et al. teaches a noise compensation algorithm that reduces the effect of 
motion blur. Regarding claim 2, in Kalivas et al., a spatiotemporal filter based on motion 
compensation (§4.2) of an image segmented into moving objects and a static 
background (§2.1) is used to reduce noise (abstract). This is the claimed first element 
of "using object motion estimation for arbitrarily shaped segments to align 
corresponding pixels between at least two frames. The spatiotemporal filter reduces 
blur (abstract, section §4) by using a weighting system (§2.1) to ensure that only moving 
pixels are motion compensated (equation 16). This is a portion of the claimed third 
element of "reducing impact of color blur from said segments that are no longer 
adjacent by adjusting weights assigned to one or more frames for pixels that lie within a 
blur region near said segment boundary". The Spatiotemporal Mean Filter defined in 
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equation 16 of Kalivas et al. is the claimed "weighted average of color values of said 
corresponding pixels" in the fourth element of the claim. 

The present invention differs from Kalivas et al. in that the present invention 
performs special coding on "segments that are no longer adjacent to a segment 
boundary based on said object motion estimation", as defined in the second element of 
the claim, or for especially "reducing the impact of color blur" from these segments by 
adjusting the weights "for pixels that lie within a blur region near said segment 
boundary" as specified in the third element of the claim. 

Prakash et al. teaches a video motion encoder and decoder. This encoder 
performs segmentation of images (pg. 5: lines 27-34) into moving objects and a 
background, as does Kalivas et al., but Prakash et al. performs special coding of 
residue data from previously-hidden regions of an image that are freshly exposed due to 
the motion of the moving objects (page 9: lines 16-29). As shown in pages 2-3 of the 18 
July 2008 Office action, these previously-hidden regions are regions "no longer adjacent 
to a segment boundary based on object motion estimation", and so determining these 
regions (page 19: line 3-page 20: line 10). This residue data may be determined as 
motionless "background residue" or moving "local residue", as determined by other 
structural information, such as the locations of boundaries and movement of moving 
objects in the frame (page 9: line 16-column 10: line 8). By properly determining 
whether this data is moving or still, the correct value of weighting function A in Kalivas is 
applied, thus reducing the impact of blur from these regions, as specified in the third 
element of the claim. 
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The Kalivas reference, then, may be considered a "base" method of filtering upon 
which the present invention is an improvement by reducing the blur from image 
segments previously adjacent to segment boundaries. The Prakash et al. reference 
demonstrates a known technique of specially determining previously-hidden segments 
of an image, considered analogous or equivalent to the claimed previously-adjacent 
segments, and properly classifying them as background or motion objects to reduce 
"residue" error data. Therefore, it would have been obvious to one having ordinary skill 
in the art at the time the present invention was made to use the residual coding 
techniques of the Prakash reference before applying the motion-compensated 
spatiotemporal filtering of the Kalivas reference to an image with the predictable result 
of more accurately segmenting an image into moving and stationary components 
(Kalivas, §1) to reduce blur, since it has been held that to apply a known technique to a 
known method ready for improvement to yield predictable results involves only routine 
skill in the art. MPEP 2143(D); Dann v. Johnston, 425 U.S. 219, 189 USPQ 257 (1976); 
In re Nilsen, 851 F.2d 1401 , 7 USPQ2d 1500 (Fed. Cir. 1988). 

Regarding claim 3, in Kalivas et al., newly-exposed pixels determined to be 
"background" residue by Prakash et al. (pg. 9: lines 16-29; pg. 19: lines 8-25; pg. 23: 
lines 9-24) are given object indicator weight A of 0 (§2.1 ). 

Regarding claim 4, Kalivas et al. and Prakash et al. are silent regarding Group of 
Picture (GOP) structure. However, the examiner takes Official Notice that the limitation 
of "determining additional motion information across GOP boundaries" is a well-known 
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part of the MPEG standard. A GOP for which motion estimation, prediction, or 
compensation can be taken from a reference frame across a GOP boundary is well- 
known in the art as an "open GOP". It would have been obvious for one having ordinary 
skill in the art at the time the invention was made to include a filter on a video stream 
having open GOPs, since open GOPs allow for reduced bandwidth in an encoded 
image sequence. 

Regarding claim 6, any apparatus that performs the filtration of Kalivas et al. as 
enhanced by Prakash et al. would perform the claimed temporal filtration as described 
in claim 2. 

Regarding claim 8, the motion compensation of Kalivas et al. comprises the 
claimed steps of "estimating motion of objects" and "aligning pixels from a current 
frame". Calculating the spatiotemporal mean filtering in Kalivas et al. is the claimed 
step of "calculating a weighted average". Determining the status of residual data in 
Prakash et al. and updating the weighting factor of this data in Kalivas et al. is the 
claimed step of "adjusting weights". 

Regarding claim 9, in Prakash et al., a background residue is the claimed 
"exposed area", and regarding claim 10, in Prakash et al., a local residue encompasses 
the claimed "region between converging objects". 

Regarding claim 11, Kalivas et al. and Prakash et al. are silent regarding Group 
of Picture (GOP) structure. However, the examiner takes Official Notice that the 
limitation of "determining additional motion information across GOP boundaries" is a 
well-known part of the MPEG standard. A GOP for which motion estimation, prediction, 
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or compensation can be taken from a reference frame across a GOP boundary is well- 
known in the art as an "open GOP". It would have been obvious for one having ordinary 
skill in the art at the time the invention was made to include a filter on a video stream 
having open GOPs, since open GOPs allow for reduced bandwidth in an encoded 
image sequence. 

10. Claims 5 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kalivas et al. in view of Prakash et al. as applied to claims 1 and 8 above, and further in 
view of US Patent 5,544,239 A (Golin et al.). The above-cited art do not teach adjusting 
a temporal filter based on a lighting offset. 

Golin et al. teaches a motion estimation method that compensates for a fading 
image. Regarding claims 5 and 12, Figure 1 of Golin et al. shows brightness 
adjustment unit 104, which calculates base image 106 by reducing pixels in an image 
by the average pixel brightness in the image and in the next image (column 2: lines 43- 
53). This is in response to fade detector 101, which stores the frame in a buffer if a 
sequence is fading (column 2: lines 25-42). Motion analysis unit 108 then determines 
displacement vectors between the current image and the previous base image (column 
3: lines 16-15). 

Kalivas, in combination with Prakash, discloses the claimed invention except for 
calculating a lighting offset. Golin et al. teaches that it was known to calculate motion 
analysis in an image sequence based on images with adjusted brightness. Therefore, it 
would have been obvious to one having ordinary skill in the art at the time the invention 
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was made to add a brightness adjustment unit to a motion analysis system as taught by 
Golin et al., since Golin et al. states in column 1, lines 41-55 that such a modification 
would increase accuracy of motion estimation. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571)272- 
9662. The examiner can normally be reached on Monday-Friday from 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 



